Results Report

Craig Brinkerhoff

2022-08-26

Note: This report contains all of the results generated by the project pipeline. Most of these results are also written to file in cache/ but for the sake of report reproducibility we pull directly from the analysis pipeline.

Background

In 2020, the Trump administration passed the ‘Navigable Waters Protection Act’ (NWPA) to “more clearly define” the waters of the United States (WOTUS). WOTUS serves as the hydrography backbone for the Clean Water Act, determining which hydrological features fall under Clean Water Act jurisdiction (and all that that entails).

The Trump-era NWPA makes clear these definitions by explicitly removing WOTUS status from ‘ephemeral streams’, those that only flow after precipitation events. In Pascua Yaqui Tribe v. U.S. Environmental Protection Agency, the legality of the NWPA was questioned and has risen to the United States Supreme Court. Because of this, implementation of the NWPA has currently halted.

Here, we model where these ephemeral streams are across the United States, and quantify the downstream impact that the NWPA would have on streamflow.

Model Overview

Model overview. Green are existing datasets, purple are our model, yellow is model robustness testing, and blue is the ultimate result/goal of the analysis.

Parameter Setup

Parameters with ranges listed will be described in greater detail in Results: Parameter Sensitivity section.

Parameter Type Nature Description Tested
Water Table Threshold Single Meta Small buffer around the free surface 10cm
Snapping Threshold Single Meta Distance buffer to assign field data to river reaches 1-50m
Non-Ephemeral Flow Frequency Threshold Single Meta Minimum % of year that non-ephemeral streamgauges can run dry 5%
Runoff ratio scalar Single Physical % scalar to perturb runoff ratio data -35%, 0%, 35%
Runoff memory Lumped Physical ‘Memory in the basin-scale runoff system’, i.e. captures delayed throughflow + delayed overland flow + general antecedent moisture conditions 0, 1, 7 dys
Minimum runoff threshold Single Physical Minimum runoff necessary to generate streamflow 0.24 mm/day

Current Caveats

  • ‘Ephemeral channels’ refers to U.S. ephemeral streams under consideration of losing WOTUS status. Canadian and Mexican ephemeral streams are not included in this category as they are outside of WOTUS status.
    • However, all Canadian and Mexican streams flowing into the U.S. are included in the model as streams not effected by the NWPA.
    • Same with canals/ditches/ponds. Included in the initial water table modeling but then excluded from final ‘ephemeral/not ephemeral classification’
  • 3 basins in Michigan/New York/Ontario and 1 basin in Montana/Alberta have no USGS gauges (and thus runoff data). I annually set their runoff terms to those for the ‘most reasonable adjacent basin’, i.e. 0904 won’t be set to the adjacent basin over the Continental Divide…
  • There is a quirk in the NHD data for the state of Indiana where 1-2 extra stream orders of river segments were added to the NHD, creating unrealisitic drainage densities in the hydrography, artifically defined by the state boundary. While this might be ok for modeling ephemeral channels, basin hydrographs are unrealistic because the basins cross the state boundary, creating way too dense network on one side and not-so-dense network on the other side. So, I manually removed stream orders until the ‘in-Indiana’ part of the river network visually matched the density of the ‘out-of-Indiana’ part of the river network. For these basins, this was generally 1 or 2 orders removed. We then did the rest of the modeling as normal.
  • Ultimately, the model was run using a runoff memory of 1 day (i.e. given a rain event, streamflow is produced for 1 day via delayed arrival of overland flow and throughflow). This was informed by common knowledge that arid ephemeral runoff memory is usually < 1 day, but in the Eastern US it has been reported as generally around 2 days (North Carolina).

Results: Validation

Mapping Ephemeral Channels

Below is a confusion matrix detailing overall ability to correctly identify ephemeral features (green is good, grey is bad). Following are classification performance metrics 

## [1] "% accuracy: 0.81"
## [1] "% balanced accuracy: 0.81"
## [1] "sensitivity: 0.8"
## [1] "specificity: 0.82"
## [1] "precision: 0.54"
## [1] "TSS: 0.62"
## [1] "F1: 0.644776119402985"

This assessment is more useful regionally, to identify trends:

## [1] "Regional average % accuracy: 0.85"
## [1] "Regional average % balanced accuracy: 0.8"
## [1] "Regional average sensitivity: 0.75"
## [1] "Regional average specificity/recall: 0.86"
## [1] "Regional average precision: 0.5"
## [1] "Regional average TSS: 0.6"
## [1] "Regional average F1: 0.6"

Individual Model Components

Below is validation of the river discharge model at streamgauges across the entire US. ‘Gage flow’ refers to the model after adjustment to truth at each gauge. ‘Ungauged flow’ refers to a more realistic validation representative of the vast majority of (ungauged) rivers, where there is no gauge to inform the model.

Number of Flowing Days

Below is a coarse verification of our estimates of basin-scale, average number of flowing days in ephemeral streams. Note that data on this property is incredibly sparse and so the best we can do is verify that the orders of magnitude are reasonable. We can then do a sensitivity analysis to confirm that orders of magnitude are sufficient (below). The purple point (in the Duke Research Forest in North Carolina) is technically measuring flow frequency in an intermittent channel that has only ephemeral streams upstream. So, it is going to be relatively underestimated by our model, as the intermittent gauged reach has some groundwater influence. This is clearly borne out in the plot.

## `geom_smooth()` using formula 'y ~ x'
## `geom_smooth()` using formula 'y ~ x'

Results: Model

CONUS wide influence of ephemeral streams

Below is the % of streamflow that passes through all CONUS ephemeral features when flowing.

## [1] 0.2089905

Below is the % of network length that is ephemeral

## [1] 0.6842736

Basin-scale influence of ephemeral streams

Below is the same result, but done out for each basin to identify regional trends (numbers, followed by boxplot).

##      Min.   1st Qu.    Median      Mean   3rd Qu.      Max. 
## 0.0002265 0.0603669 0.1080814 0.2315782 0.2960696 0.9996894

Interpret this color scale with a lot of caution… It is automatically generated and not idea.

Results: Parameter Sensitivity

There are specific concerns about model parameters and limitations of process representation at this scale, so set of parameter sensitivity tests are needed. For each one, I outline the goal of the test, the specifics of the test, the expected result if the model is working as hoped, and then the figures plotting the results.

Ephemeral Mapping Sensitivity

Goal: Make sure classification accuracy isn’t wildly dependent on snapping threshold. It is expected that many of these field assessments were made in streams not actually represented in the NHD, and so we need to determine a reasonable buffer around each river to account for both locational errors and data that shouldn’t actually be snapped to the river network.

Approach: Compare ephemeral mapping accuracy to the snapping threshold

Expected: As snapping threshold increases, accuracy should decrease but also asymptote to a value that will be a bit of an underestimate (given the rational provided in the next test).

Ephemeral Scaling Sensitivity

Goal: Check which snapping thresholds yield patterns and scaling relationships that are expected ‘in the wild’. Again, it is expected that many of these field assessments were made in streams not actually represented in the NHD, and so we need to determine a reasonable buffer around each river to account for both locational errors and data that shouldn’t actually be snapped to the river network.

Approach: Using the EPA data, check which snapping thresholds yield expected classical/Hortonian scaling across all field-verified ephemeral channels, assuming that Horton’s laws hold across all ephemeral channels. This is done by comparing the snapping threshold to 1) strength-of-fit of classical/Hortonian river network scaling theory of ephemeral channels (orange) and 2) the number of orders that would need to be additionally scaled to yield the observed distributions in the EPA WOTUS data (green). - Green line: because much of this data is on streams/creeks smaller than those in the NHD, we 1) Fit the horton law of stream numbers to the EPA/Corps data 2) Use this equation to find the stream order at which number of streams matches the EPA/Corps data NOT on the NHD. This will be the order that we ‘scale down to’, assuming that the EPA/Corps data is of sufficient resolution to include the smallest ephemeral channels out there. This is conservative and we are likely still missing some streams. 3) Calculate MAE across all orders (existing + scaled orders) to determine which snapping thresholds ‘best fit the data’. MAE is mean absolute error, with smaller values being ‘better’.

Expected result: As we increase the snapping threshold, it is expected that MAE (orange) will initially decrease as any locational errors are handled, but then begin increasing again as we begin mis-assigning EPA field data to the wrong rivers. There should also be a range of snapping thresholds where the number of scaled stream orders (green) is reasonably robust. Artifically high or low snapping thresholds should yield sporadic # of scaled orders, while resonable threshlds should all yield ~the same number.

This analysis is very much in the weeds, but basically we are looking for small orange values and a green value that is insensitive to changes in the snapping threshold. We also use the previous plot to inform this analysis. This is about 10-15m.

Number Flowing Days Sensitivity

Goal: Confirm that reasonable order of magnitude estimates of ‘number of flowing days’ are sufficient.

Approach: Re-run the model under a few runoff scenarios that get increasingly extreme and unlikely. These tests are detaled below:

  1. ‘Low-runoff scenario’: runoff ratio reduced by 33% and runoff memory of 0 days. Representative of low antecedent moisture conditions.
  2. ‘High-runoff scenario’: runoff ratio increased by 33% and runoff memory of 10 days. Representative of high antecedent moisture conditions.

Expected Result: Little change in final model results given these runoff perturbations.

*‘Calibrating’ a streamflow threshold

##    thresh       mae
## 1  0.0010 22.122615
## 2  0.0025 20.637660
## 3  0.0050 19.058463
## 4  0.0075 17.881771
## 5  0.0100 16.928126
## 6  0.0250 13.295910
## 7  0.0500  9.914923
## 8  0.0750  8.137049
## 9  0.1000  7.132949
## 10 0.2500  5.889295
## 11 0.5000  7.543952
## 12 0.7500  9.519565
## 13 1.0000 11.100380